Posts Tagged: #Markov Chain

Convergence of Markov Chain

## What is Markov Chain ? Markov Chain is a Stochastic Model in which Future is dependent only on Present not on Past , What I mean to say that is $$ P(X^{t+1}|X^t,X^{t-1},...X^2,X^1) = P(X^{t+1}| X^t) $$ #### Transition Probability Matrix Let us denote $$ p_{ij} = P(X^{n+1} =...